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CONTROLLING OPERATION OF A DEVICE 
USING A RE-CONFIGURABLE 
WATERMARK DETECTOR 

RELATED APPLICATION DATA 

This application is a continuation of co-pending applica- 
tion Ser. No. 09/574,668, filed May 18, 2000, which claims 
priority to provisional patent application No. 60/134,782, 
filed May 19, 1999, incorporated herein by reference. 

The subject matter of the present application is related to 
that disclosed in application Ser. No. 09/234,780, filed Jan. 
20, 1999; Sen No. 09/314,648, filed May 19, 1999; Ser. No. 
09/337,590, filed Jun. 21, 1999; Ser. No. 09/433,104, filed 
Nov. 3, 1999; Ser. No. 09/441,819, filed Nov. 17, 1999; Ser. 
No. 09/441,821, filed Nov. 17, 1999; Ser. No. 09/442,441, 
filed Nov. 17, 1999; Ser. No. 09/464,307, filed Dec. 15, 
1999; Ser. No. 09/473,075, filed Dec. 28, 1999; Ser. No. 
09/476,686, filed Dec. 30, 1999; Ser. No. 09/482,752, filed 
Jan. 13, 2000; Ser. No. 09/484,742, filed Jan. 18, 2000; Ser. 
No. 09/498,223, filed Feb. 3, 2000; No. 60/180,364, filed 
Feb. 4, 2000; Ser. No. 09/520,406, filed Mar. 8, 2000; Ser. 
No. 09/563,664, filed May 2, 2000, and Ser. No. 09/562,517, 
filed May 1, 2000. 

FIELD OF THE INVENTION 

The present invention relates to applications of digital 
watermarking in conjunction with audio, video, imagery, 
and other media content. 

BACKGROUND AND SUMMARY 

Watermarking (or "digital watermarking") is a quickly 
growing field of endeavor, with several different approaches. 
The present assignee's work is reflected in U.S. Pat. No. 
5,862,260; in copending application Ser. Nos. 09/503,881 
and 09/452,023; and in published specifications WO 
9953428 and WO0007356 (corresponding to U.S. Ser. Nos. 
09/074,034 and 09/127,502). A great many other approaches 
are familiar to those skilled in the art. The artisan is 
presumed to be familiar with the full range of watermarking 
literature. 

In the present disclosure it should be understood that 
references to watermarking encompass not only the assign- 
ee's watermarking technology, but can likewise be practiced 
with any other watermarking technology. 

The invention relates to methods, devices and systems for 
controlling operation of a device using a re -configurable 
watermark detector. In many applications, it is useful to be 
able to change the operation of a watermark detector. Such 
changes may include changing how the watermark detector 
decodes or interprets a watermark embedded in a signal of 
a given media type, such as audio, video or still images. 

The ability to reconfigure a watermark detector is advan- 
tageous in many applications. When watermark detectors 
are deployed widely in consumer electronic devices and 
software, the capability to reconfigure the watermark detec- 
tor reduces or prevents the devices and software from 
becoming obsolete when changes to the detector are needed. 
In content management schemes, where the watermark is 
used as a security feature, the security mechanism may be 
hacked. In other applications, the watermark technology 
provider may want to incorporate new features into the 
watermark detector, expand the message payload of the 
watermark, or change how the watermark payload is inter- 
preted. In these circumstances, it is advantageous to be able 
to reconfigure a watermark detector, and particularly, to 
reconfigure the detector remotely. 
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One aspect of the invention is a method for controlling the 
operation of a device using a watermark. The method 
decodes a watermark embedded in a signal of a given media 
type, such as an image or audio signal (e.g., still images, 
motion pictures, audio, video, etc.). The watermark includes 
a command signal used to trigger a change in operation of 
the watermark detector. Based on the command signal, the 
method changes the operation of the watermark detector. 
This change may include changing how the watermark 
detector decodes or interprets a watermark in a signal of the 
media type. After changing operation of the watermark 
detector, the method decodes a usage control restriction 
from a watermark embedded in the signal. 

The scope of this method encompasses a variety of 
implementations. The command signal may be represented 
as one or more bits of a watermark payload carried by the 
watermark. Changing the operation of the detector may 
include re -programming it or altering how it interprets 
watermark data embedded in a media object. For example, 
the method may transfer firmware instructions to the detec- 
tor to replace instructions stored earlier. As another example, 
the command may change the operation of the detector 
according to a preprogrammed rule. The rule may define a 
change in watermark key, for instance. Yet another example 
is changing the operation of the detector by changing how a 
device responds to the watermark signal extracted from a 
media object. The behavior of a hardware or software media 
player, for instance, may be updated to respond differently to 
the watermark signal, and specifically, to the message car- 
ried in a watermark payload. 

The method may use watermark payload data to change 
the operation of the detector. For example, the payload may 
specify instructions or watermark key data. In response to 
this payload data, the detector may install and execute new 
instructions or use the new watermark key to decode water- 
marks in media objects. 

Another aspect of the invention is an alternative method 
for controlling the operation of a device using a watermark 
detector. This method receives a media object and a com- 
mand associated with the media object signaling that the 
40 watermark detector requires an upgrade to decode a water- 
mark from the media object. In response to the command, 
the method updates the watermark detector to create an 
updated watermark detector. It decodes the watermark from 
the media object with the updated watermark detector, 
including decoding a usage control restriction from the 
watermark. The command may be encoded in a watermark 
in the media object, or may be conveyed in a channel 
different than the watermark channel yet transmitted along 
with the media object (e.g., out-of-band channels like a file 
header or footer, sub-titling data channel, SCA channel, 
etc.). The scope of this method encompasses a variety of 
implementations. In addition, aspects of this method may be 
used in combination with aspects of the method summarized 
earlier. 

Yet another aspect of the invention is a re -programmable 
watermark detector. The detector comprises a watermark 
decoder for detecting a command to upgrade the detector. It 
also includes instructions that are replaceable in response to 
detecting the command to upgrade the detector. The detector 
instructions are operable to decode a usage control restric- 
tion from a watermark and control playback or recording of 
a media object carrying the watermark. The instructions may 
be conveyed to the watermark detector along with a media 
object, either in a watermark payload, or a channel other 
than the watermark channel. 

Another aspect of the invention is a method of an encod- 
ing an upgrade trigger in a watermark. The method receives 
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a media object of a given media type, and encodes a 
watermark into the media object. The watermark includes a 
command signal used to trigger a change in operation of a 
watermark detector. When received in a detector, this change 
operates to alter how the watermark detector decodes or 5 
interprets a watermark in a signal of the media type. In 
addition, the method encodes a usage control restriction into 
the watermark, where the chaDge in the operation of the 
detector enables the watermark detector to read the usage 
control restriction. io 

Further features and aspects of the invention will become 
apparent with reference to the following detailed description 
and accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 15 

FIG. 1 is a diagram showing the participants, and 
channels, involved in the distribution of music. 

FIG. 2 shows a conceptual model of how music artists, 
record labels, and E-Music distributors can all interact with 2 o 
a Media Asset Management System, of which several are 
detailed in the following specification. 

DETAILED DESCRIPTION 

For expository convenience, much of the following dis- 25 
cussion focuses on music, but the same principles and 
techniques are largely or wholly applicable to other source 
data, whether non-music audio, video, still imagery, printed 
materials, etc. 

Music Asset Management 30 

Referring to the figures, the music distribution process 
begins with a creative artist 10. The artist's music has 
traditionally been distributed by a record label 12. (While 
the following discussion refers to distribution through such 
a label, it should be understood that such distribution can 35 
just as well be effected directed under the artist's control, 
without a record label intermediary.) 

In traditional distribution 14, the record label produces 
tangible media, such as records, tapes, videos (e.g. music 
videos), and CDs 16. These media are physically distributed 40 
to end-consumers 18. Additionally, the label 12 distributes 
the music media to outlets 20, such as radio and TV stations, 
cable and satellite systems, etc., which broadcast (or 
narrowcast) the artist's work to an audience. Distribution 
through such media outlets may be monitored by playout 45 
tracking services. Playout tracking data, collected by firms 
including Arbitron, Nielsen, AS CAP, BMI, etc., can be used 
to compute royalty payments, to verify broadcast (e.g. for 
advertising), etc. 

Increasingly, the distribution of the music to the media 50 
outlets is performed electronically. Such distribution first 
took the form of analog audio over high quality landlines or 
satellite channels. Digital audio quickly supplanted analog 
audio in such distribution channels due to higher fidelity. 

More recently, distribution of the music from the record 55 
labels to the media outlets has occurred over secure links, 
now including the internet. Such security was first provided 
simply by scrambling the audio signal or data. More sophis- 
ticated "container" -based systems are now coming into 
vogue, in which the audio is "packaged" (often in encrypted 60 
form) with ancillary data. 

Electronic distribution of music to the consumer is also 
gaining popularity, presently in the MP3 format primarily. 
The music providers may deal directly with the public, but 
more commonly effect such consumer distribution through a 65 
newly emerging tier of digital media outlets, such as internet 
sites that specialize in music. From such sites, consumers 



can download digital audio files into personal digital audio 
players. (The Diamond Rio, and the Audible MobilePlayer 
devices are some of the first of what will doubtless be a large 
number of entrants into this personal internet audio appli- 
ance market.) Or the downloaded data can be stored by the 
consumer-recipient onto any other writeable media (e.g. 
hard disk, CD, DVD, tape, videotape, etc.). Typically a • 
personal computer is used for such downloading, but this 
intermediary may be dispensed with by coupling next gen- 
eration of personal audio appliances to an internet-like link. 

The data downloaded by the consumer can be stored 
either in the native digital format, translated into another 
digital format (which translation may include decryption), 
converted into analog and recorded in analog form, etc. 

Unauthorized copying or use of the music can occur 
anywhere in the foregoing channels. However, one of the 
greatest risks occurs once the music has been delivered to 
the consumer (whether by tangible media, by traditional 
broadcast media outlets, by emerging digital distribution, or 
otherwise). 

The general idea of embedding auxiliary data into music 
(i.e. watermarking) has been widely proposed, but so far has 
been of limited applicability. 

For example, GoodNoise is planning to embed a digital 
signature — termed a multimedia identifier, or MMI — in its 
MP3 music. MMI will register the song and its author with 
a licensing number. In addition to providing information 
about the songwriter and distributor, this digital encoding 
may also include lyrics, liner notes, and other information. 
But all of the proposed uses serve only to convey informa- 
tion from the distributor to the consumer; use for "tracking" 
is actively disclaimed. (Wired News, "GoodNoise Tags MP3 
Files," Feb. 3, 1999.) 

The Genuine Music Coalition — a partnership of various 
companies in the music distribution business — likewise has 
announced plans to employ watermarking of MP3 music. 
The watermarking technology, to be provided by Liquid 
Audio, will convey data specifying the artist or producer 
contact, copyright data, and a number to track ownership. 
The Coalition hopes that the provision of this embedded 
information will help thwart piracy, Industry observers 
believe Liquid Audio will next introduce playback technol- 
ogy only plays audio in which its watermark is detected. 
(Wired News, "Liquefying MP3," Jan. 23, 1999.) 

A similar initiative has been announced by the Recording 
Industry Association of America (RIAA). Termed the Secure 
Digital Music Initiative (SDMI), the program seeks to define 
a voluntary specification that will assure proper compensa- 
tion to those who produce and distribute music. One element 
of the system will likely be a watermarking component. 
(Dow Jones Newswire, "Spurred By Maverick Technology, 
Music Industry Eyes Web," Dec. 31, 1998.) 

Yet another initiative has been announced by Solana and 
ASCAP. Other companies promoting watermarking for 
music include Aris Technology, MCY.com, and AudioSoft. 

The watermark payload can represent various types of 
data. An exemplary payload includes data relating to the 
artist, distribution entity, title, and copyright date/proprietor. 
Additionally, the payload can include a digital object 
identifier — an ISBN-like number issued by a central orga- 
nization (e.g. a rights management organization) to uniquely 
identify the work. 

Such payload data can be encoded literally (e.g. the title 
by a series of ASCII characters, etc.). In other embodiments, 
codes or abbreviations can be employed — with each code 
having a known meaning. In still other embodiments, the 
data can be meaningless by itself, but may serve as a key 
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(e.g., a Unique Identifier, or UID) into a remote data corruption. By identifying such known signal, the water- 
database or repository. An example of such a remote data mark signal can then be decoded. (The known signal can 
repository is a web site at a Master Global Address (MGA) take various forms, e.g. a synchronization signal, a marker 
associated with content, as detailed below. signal, calibration signal, a universal code signal as 

An exemplary data payload may, for example, have the s described in applicant's patents, etc.) 

following format: * n some embodiments, a watermark "dial-tone" signal is 

provided. This dial-tone signal is a low amplitude, relatively 

_ — . — — — — — — wideband, repetitive signal that commonly conveys only 

| A | B|C | D|B | F | 0|H | I | | | timited information (e.g. a single bit of information). Its 

1Q presence in an audio signal can serve as a "do not record," 

„„ A . ■ i_ * /o u- 4 . i 4 \ AOniT t . . or similar instruction signal. Alternatively, or in addition, the 

WhereAisasix-b^ s[ { can ^ as an M \ t ' Q a 

as a digital object identifier (which may serve as a link to a . phlfal _ bit digital watermark si ^ that fa also encoded in the 

Master Global Address through a default name server, as audio For example) the cyclical repe tition of the signal can 

discussed below), B is a two-byte ASCII field serving as a scrvc t0 idcntifv the start of mc plural . bit digita] wa t C rmark 

key into an "artist" field of the remote database, C is a 15 s j gna l. Or the spectrum or repetition rate of the signal can 

three-byte ASCII field serving as a key into a "title" field of identify any temporal corruption of the audio. An exemplary 

the remote database; D is a 14-bit field serving as a key into such signal is detailed as a "simple universal code" in U.S. 

a "label" field of the remote database, E is an 8-bit integer Pat. No. 5,636,292. 

representing the work's year of first publication (with 0 A track of music can be pre-authorized for specified types 
representing the year 2000); F is a 10-bit field serving as a 20 of use. For example, the usage control string of the water- 
key into a "price" field of the remote database, G is a mark payload may include a six-bit field detailing the classes 
two-byte usage control string (detailed below), H is a of devices for which the audio is authorized. Each bit would 
streaming data channel, and I is a string of bits serving as a correspond to a different class of device. Class 1 devices 
cyclic redundancy checksum for the foregoing. (More may be personal playback devices with only analog-audio 
sophisticated error correcting checksums can, of course, be 25 output. Class 2 devices may be personal entertainment 
employed.) This payload format totals 136 bits, exclusive of devices capable of outputting music in digital (e.g. MP3, 
the CRC coding and the streaming data channel. redbook, *.WAV) format, as well as analog audio. Class 3 

This payload is encoded repeatedly, or redundantly devices may be personal computer systems (i.e. with essen- 

through the music, so that the full payload can be decoded tially unlimited ability for processing and outputting digital 

from partial excerpts of the music. 30 audio). Etc., etc. A device to which such MP3 audio is 

The encoding is also desirably perceptually adaptive, so provided would check the usage control string data to 

that higher energy encoding is employed where the listener determine whether it is authorized to utilize the audio. A 

is less likely to perceive the additional "noise" introduced by personal playback device with analog-only output, for 

the encoding, and vice versa. Various techniques for per- example, would examine the first bit of the usage control 

ceptually adaptive encoding are known. For example, some 35 string. If it was "1," the device would be authorized to use 

tie the amplitude of the encoded signal to the instantaneous (i.e. playback) the MP3 data; if it was a "0," the device 

amplitude of the music. Others exploit psycho acoustic would refuse to play the music. 

"masking" of one signal by a spectrally- or temporally- In addition to pre -authorization for certain classes of 

adjoining signal of higher energy. Still other approaches fill devices, the usage control string can also include bits 

gaps in the music's spectrum with watermark energy. 40 indicating the number of permitted playbacks. This data can 

In other embodiments, perceptually adaptive encoding is be encoded in bits seven through nine, representing eight 

not used. In some such embodiments, no tailoring of the possibilities: 

temporal or spectral characteristics of the watermark signal 0 — no playback permitted 

is employed. In others, the watermark signal is spectrally 1— single playback permitted 

filtered to emphasize low frequency audio components (e.g. 45 2^-two playbacks permitted 

less than 500 hz), high frequency audio components (e.g. 3_l lnree playbacks permitted 

higher than 2500 hz), or mid-frequency audio components A . , , . , 

(500-2500 hz). ^" four P la y backs Pitted 

The streaming data field channel (H) is a medium by 5 ~ five Paybacks permitted 

which data can be conveyed from a distribution site (or other 50 6 10 playbacks permitted 

site) to the end user. Such data may be entirely unrelated to 7— unlimited playbacks permitted 

the underlying work. For example, it may serve a utilitarian 8 — refer to associated data (within the watermark, or 

purpose, such as conveying data to a memory in the con- stored at a remote site) which specifies number of 

sumer device to replace previously-stored data that is out- permitted playbacks. 

of-date. It may be a commercial channel on which band- 55 The playback device may include a non-volatile store in 

width is sold for access to the consumer or the consumer's which the number of permitted playbacks is stored for each 

device. Essentially any purpose can be served by this track of music. The device would decrement this number at 

streaming data field. Unlike most of the other fields, the the beginning of each playback. 

streaming data field may not endlessly repeat the same data, The usage control string can also include a two-bit field 

but can convey data that changes with time. 60 (bits ten and eleven) indicating recording permissions. A 

Desirably, the encoding is performed in a manner permit- value of 0 means that data corresponding to the MP3 audio 

ting recovery of the watermark data even if the audio is (regardless of digital format) should never be made available 

corrupted, e.g. by format conversion, re-sampling, tape wow to another digital device. A value of 1 means that the data 

and flutter, compression, coding, or various forms of audio corresponding to the MP3 data may be made available once 

processing (e.g. filtering, p re-emphasis, re -scaling, etc.). 65 to another digital device. A value of 2 means that the data 

One way to provide for such robustness is to encode a signal may be made available an unlimited number of times to 

of known character that can be recognized through all such other digital devices. (Value 3 is reserved.) 
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Another data field that can be included in an audio cassette/radio players. However, the desired music was 

watermark is a rating that indicates age-appropriateness. sometimes difficult to encounter in a radio broadcast, and the 

Music with violence or sexual themes might be given a quality was less than superb. 

rating akin to the MPAA "PG-13" or "R" rating. Audio The combined cassette/radio player has now evolved into 
appliances may be programmed to recognize the rating of 5 a general purpose computer with wide-ranging functionality, 
incoming music, and to interrupt playback if the rating and other sophisticated devices. Music can be acquired off 
exceeds a certain threshold setting. Various known tech- the web, and can be recorded in various forms (e.g. in a 
niques can be employed to assure that such settings cannot personal MP3 player, stored on a hard disk, stored on a 
readily be changed, e.g., by juvenile listeners. writeable CD-ROM, played back and recorded on analog 
Another data field that can be included in an audio 10 cassette, etc., etc.). The quality can be quite high, and the 
watermark is a date field. This field can indicate either the erratic broadcast time problems of radio broadcasts have 
date the music was watermarked, or a date in the future on been overcome by the web's on-demand delivery mecha- 
which certain rights associated with the music should nisms. (Moreover, the music can be downloaded in faster- 
change. Some consumers, for example may not wish to than-realtime, a further benefit over recording-off-the-air 
purchase perpetual playback rights to certain musical selec- 15 techniques.) 

tions. The right to play a selection for 6 months may suffice One hybrid between the new and old is a novel radio (e.g., 

for many consumers, especially if the price is discounted in for use in a car) that has a "capture" button on the front panel 

view of the limited term. Such an arrangement would not be (or other form of user interface, e.g., a Capture icon on a 

wholly disadvantageous to music distributors, since some GUI). If a user hears a song they want to record and keep, 

consumers may end up purchasing music twice if their initial 20 they press the Capture button while the song is playing. In 

assessment of a musical selection's appeal was too short- response, the radio device decodes a watermark embedded 

sighted. (Naturally, the playback equipment would require a in the music, and thereby knows the identity of the music, 

source of real-time clock data against which the date field in The radio then makes a wireless transmission identifying the 

the watermark can be checked to ensure that the playback user and the desired song. A local repeater network picks up 

rights have not yet expired.) 25 the wireless signal and relays it (e.g. by wireless rebroadcast, 

Another of the data fields that can be included in an audio by modem, or other communication medium) to a music 

watermark specifies technical playback parameters. For clearinghouse. The clearinghouse charges the user a nominal 

example, the parameter can cause the playback appliance to fee (e.g. via a pre-arranged credit card), and queues the 

apply a spectral equalization that favors bass frequencies, or music for download to a predetermined location associated 

treble frequencies, or mid-range frequencies, etc. Other 30 with the user. 

p re-configured equalization arrangements can similarly be In one embodiment, the predetermined location is the 

invoked responsive to watermark data. Likewise, the param- user's own computer. If a "live" IP address is known for the 

eter can invoke special-effects provided by the playback user's computer, the music can be transferred immediately, 

appliance, e.g., echo effects, reverb, etc. (Again, such If the user's computer is only occasionally connected to the 

parameters are usually represented in an abbreviated, coded 35 internet, the music can be stored at a web site (e.g. protected 

form, and are interpreted in accordance with instructions with a user-set password), and can be downloaded to the 

stored in a memory (either in the playback appliance, or user's computer whenever it is convenient, 

linked thereto). In other embodiments, the predetermined location is a 

The same data fields and principles can be applied to personal music library maintained by the user. The library 

non-audio content. In video, for example, watermarked data 40 can take the form, e.g., of a hard-disk or semiconductor 

can adaptively control the display monitor or playback memory array in which the user customarily stores music, 

parameters (e.g., color space) to enhance the viewing expe- This storage device is adapted to provide music data to one 

rience. or more playback units employed by the user (e.g. a personal 

Music Asset Management/Commerce MP3 player, a home stereo system, a car stereo system, etc.). 

The majority of domestic music piracy is not organized. 45 In most installations, the library is physically located at the 

Rather, it is a crime of opportunity and convenience. If the user's residence, but could be remotely sited, e.g. consoli- 

crime were made more difficult, the alternative of obtaining dated with the music libraries of many other users at a 

a copy through legitimate channels would be less onerous. central location. 

Similarly, if the' procedure for obtaining a copy through The personal music library can have its own internet 
legitimate channels were simplified, the incentive for piracy 50 connection. Or it can be equipped with wireless capabilities, 
would be reduced. Watermarking facilitates both — making permitting it to receive digital music from wireless broad - 
the crime more difficult, and making legitimate music acqui- casts (e.g. from the clearinghouse). In either case, the library 
sition easier. can provide music to the user's playback devices by short- 
Consider, for example, the pricing of music in conven- range wireless broadcast, 
tional record stores. A CD (compact disk) may cost $15, but 55 By such arrangement, a user can conveniendy compile an 
its sale may be driven by just one or two popular songs on archive of favorite music— even while away from home, 
the disk To obtain these songs, the consumers must pur- Many variants of the foregoing are of course possible. The 
chase the entire disk, with perhaps a dozen songs of no radio can be a portable unit (e.g. a boombox, a Walkman 
particular, interest. This, in essence, is a tying arrangement radio, etc.), rather than an automotive unit. The UI feature 
that benefits the record labels while prejudicing the consum- 60 employed by the user to initiate capture a musical selection 
ers. Given these circumstances, and a ready opportunity to need not be a button (physical or on-screen). For example, 
make copies, it is not surprising that customers sometimes in some embodiments it can be a voice-recognition system 
make illicit copies. that responds to spoken commands, such as "capture" or 

One classic technique of avoiding purchase of a complete "record." Or it can be a form of gesture interface, 

collection of music, when only one or two songs is desired, 65 Instead of decoding the watermark only in response to the 

is to record the music off the radio. While of dubious user's "capture" command, the radio can decode watermarks 

legality, this technique was popular in the era of combined from all received programs, and keep the most recent in a 



11/25/2003, EAST Version: 1.4.1 



US 6,442,285 B2 



10 



small FIFO memory. By such arrangement, the user need not 
issue the capture instruction while the song is playing, but 
can do so even after the song is finished. 

In some embodiments, data corresponding to the water- 
mark can be made available to the user in various forms. For 
example, it can be presented to the user on an LCD screen, 
identifying the artist and song currently playing. If a corre- 
sponding UI button is activated, the device can so-identify 
the last several selections. Moreover, the data need not be 
presented to the user in displayed form; it can be annunci- 
ated by known computer-speech technologies instead. 

In embodiments in which the watermark does not convey 
ASCII text data, but instead conveys ULIDs, or coded 
abbreviations, the device must generally interpret this data 
before presenting it to the user. In an illustrative 
embodiment, the device is a pocket-sized FM radio and is 
equipped with a 1 megabyte semiconductor non-volative 
RAM memory. The memory includes a data structure that 
serves as a look-up table, matching code numbers to artist 
names and song titles. When the user queries the device to 
learn the identify of a song, the memory is indexed in 
accordance with one or more fields from the decoded 
watermark, and the resulting textual data from the memory 
(e.g. song title and artist) is annunciated or displayed to the 
user. 

In most applications, such memory will require frequent 
updating. The RF receiver provides a ready mechanism for 
providing such updated data. In one embodiment, the radio 
"awakens" briefly at otherwise idle moments and tunes to a 
predetermined frequency at which updated data for the 
memory is broadcast, either in a baseband broadcast 
channel, or in an ancillary (e.g. SCA) channel. 

In variants of the foregoing, internet delivery of updated 
memory data can be substituted for wireless delivery. For 
example, the artist/song title memory in the personal player 
can be updated by placing the player in a "nest" every 
evening. The nest (which may be integrated with a battery 
charger for the appliance) can have an internet connection, 
and can exchange data with the personal device by infrared, 
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house attends to delivery of the music in a desired manner 
to the specified recipient. 

Still further, a listener may query the appliance (by voice, 
GUI or physical button, textual, gesture, or other input) to 
identify CDs on which the then-playing selection is 
recorded. Or the listener may query the appliance for the 
then-playing artist's concert schedule. Again, the appliance 
can contact a remote database, relay the query, and forward 
data from the watermark pay load identifying the artist 
and/or song title to which the query relates. The database 
locates the requested data, and relays same back to the 
appliance for presentation (via a display, by machine speech, 
or other output) to the user. If desired, the user can continue 
the dialog with a further instruction, e.g., to buy one of the 
CDs on which the then-playing song is included. Again, this 
instruction may be entered by voice, GUI, etc., and dis- 
patched from the applicance to the clearinghouse, which can 
then complete the transaction in accordance with pre-stored 
information (e.g. credit card account number, mailing 
address, etc.). A confirming message is relayed to the 
appliance for presentation to the user. 

While the foregoing transactions require a link to a remote 
site or database, other watermark-based consumer services 
can be provided without such a link. For example, a user can 
query the appliance as to the artist or song-title of the 
selection currently playing. The appliance can consult the 
embedded watermark data (and optionally consult a memory 
to determine the textual names associated with coded water- 
mark data), and provide the requested information to the 
user (e.g., by a display, annunciation, or other output). 

The foregoing concepts (e.g. Capture, Send, etc.) can also 
be employed in connection with internet — rather than radio - 
delivery of music. (The following discussion is illustrated 
with reference to the "Capture" function, but it will be 
recognized that the other earlier-discussed features can be 
similarly implemented.) 

There are many commercial web sites that sell audio (in 
CD form or otherwise), and offer limited free music 
downloads, (or music clips) as an enticement to lure con- 



inductive, or other proximity-coupling technologies, or 40 sumers. But there are also a great number of music web sites 



through metal contacts. Each evening, the nest can receive 
an updated collection of artists/song titles, and can re -write 
the memory in the personal device accordingly. By such 
arrangement, the watermark data can always be properly 
intepreted for presentation to the user. 

ITie "Capture" concepts noted above can be extended to 
other functions as well. One is akin to forwarding of email. 
If a consumer hears a song that another friend would enjoy, 
the listener can send a copy of the song to the friend. This 
instruction can be issued by pressing a "Send" button, or by 
invoking a similar function on a graphical (or voice- or 
gesture-responsive) user interface. In response, the appli- 
ance so-instructed can query the person as to the recipient. 
The person can designate the desired recipient(s) by typing 
in a name, or a portion thereof sufficient to uniquely identify 
the recipient. Or more typically, the person can speak the 
recipient's name. As is conventional with hands-free vehicle 
cell phones, a voice recognition unit can listen to the spoken 
instructions and identify the desired recipient. An "address 
book"-like feature has the requisite information for the 
recipient (e.g., the web site, IP address, or other data 
identifying the location to which music for that recipient 
should stored or queued, the format in which the music 
should be delivered, etc.) stored therein. In response to such 
command, the appliance dispatches instructions to the 
clearinghouse, including an authorization to debit the send- 
er's credit card for the music charge. Again, the clearing- 
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that have no commercial pretense. They are hosted by music 
lovers strictly for the enjoyment of other music lovers. When 
music is downloaded from such a web site, the end-user's 
computer can analyze the digital data to decode watermark 
data therefrom. Again, the user can be presented with a 
"Capture" button that initiates a commercial transaction, by 
which a complete copy of the then-downloaded audio is sent 
to a prearranged storage location, and the user's credit card 
is debited accordingly. This transaction can occur indepen- 
dently of the site from which the music is downloaded (e.g. 
through the clearinghouse referenced above). 

While the "Capture" button can be presented on the 
web-site, this would generally not be in keeping with the 
non-commercial nature of such web sites. Instead, in an 
exemplary embodiment, the Capture feature is a software 
program resident at the user's computer. When this software 
program is invoked by the user, a socket channel is instan- 
tiated between the user's computer and the clearinghouse 
over the then-existing internet connection. The decoded 
watermark data and user ID is transmitted to the clearing- 
house over this channel, without interrupting the user's other 
activity (e.g. downloading music from the non-commercial 
web site). In response, the clearinghouse transmits the music 
to the prearranged location and attends to billing. 

In some embodiments, a watermark detector is included 
as part of the operating system, and constantly monitors all 
TCP/IP, or other internet, data received by the user's 
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computer, for the presence of watermarks. In such case, 
when the Capture feature is invoked, the program examines 
a memory location in which the operating system stores the 
most-recently received watermark data. In another 
embodiment, the computer does not monitor all internet 
traffic for embedded watermark data, but includes an API 
that can be called by the Capture program to decode a 
watermark from the data then being received. The API 
returns the decoded watermark data to the Capture program, 
which relays same to the clearinghouse, as above. In still 
another embodiment, the watermark decoder forms part of 
the Capture program, which both decodes the watermark 
and relays it to the clearinghouse when the Capture program 
is invoked by the user. 

There are various techniques by which the Capture pro- 
gram can be selectively invoked. One is by a keyboard 
macro (e.g. by a combination of keyboard keys). Another is 
by a program icon that is always presented on the screen, and 
can be double-clicked to activate. (Again, confirmation 
processes may be called for, depending on the likelihood of 
inadvertent invocation.) Many other techniques are likewise 
possible. 

In the just-contemplated scenario, the Capture operation 
is invoked while the user is downloading music from a 
non-commercial web site. This seems somewhat redundant, 
since the downloading — itself — is transferring music to the 
user's computer. However, the Capture operation provides 
added value. 

In the case of streaming audio, the audio is not typically 
stored in a location in which it can be re-used by the 
consumer. It can be listened-to as delivered, but is then gone. 
Capturing the audio provides the user a copy that can be 
played repeatedly. 

In the case of downloaded music files, the music may 
have been encoded to prevent its recordal on other devices. 
Thus, while the user may download the music onto a desktop 
computer, copy-prevention mechanisms may prevent use of 
that file anywhere else, e.g. on a portable music appliance. 
Again, Capturing the audio provides the user a copy that can 
be transferred to another device. (The music file provided by 
the clearinghouse can have copy-prevention limits of its 
own — e.g., the file can be copied, but only once, or the file 
can be copied only onto devices owned by the user.) 

(Confirmation of device ownership can be implemented in 
various ways. One is to identify to the clearinghouse all 
music devices owned by a user at the time the user registers 
with the clearinghouse (supplemented as necessary by later 
equipment acquisitions). Device IDs associated with a user 
can be stored in a database at the clearinghouse, and these 
can be encoded into the downloaded music as permitted 
devices to which the file can be copied, or on which it can 
be played.) 

The commerce opportunity presented by non-commercial 
music web-sites is but one enabled by digital watermarks. 
There are many others. 

To take one example, consider the media by which music 
and artists are presently promoted. In addition to radio 
airtime, these include music videos (a la MTV), fan 
magazines, web advertisements, graphical icons (e.g. the 
Grateful Dead dancing bears), posters, live events, movies, 
etc. Watermarked data can be used in all such media as a link 
in a commercial transaction. 

A poster, for example, typically includes a photo of the 
artist, and may comprise cover-art from a CD. The photo/art 
can be digitally watermarked with various types of data, e.g., 
the artist's identify, the record label that distributes the 
artist's work, the music project being particularly promoted 
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by the poster (e.g. a CD, or a concert tour), a fan web-site 
related to the artist, a web -site hosted by the record label for 
selling audio in CD or electronic form, a web-site from 
which free music by the artist can be downloaded, data 
identifying the poster itself, etc. 

A user, equipped with a portable appliance that merges the 
functions of palmtop computer and digital camera, can snap 
an image of the poster. The processor can decode the 
watermarked data, and initiate any of various links based on 
the decoded data. 

In an exemplary embodiment, after snapping the picture, 
the user invokes a software program on the device that 
exposes the various links gleaned from the snapped image 
data. Such a program can, for example, present the option of 
linking to the artist's fan web site, or downloading free 
streaming audio or music clips, or ordering the promoted 
CD, or requesting the above-noted clearinghouse to down- 
load a personal copy of selected song(s) by the artist to the 
user's personal music library, etc. (The device is presumed 
to have a wireless internet link. In devices not having this 
capability, the requested actions can be queued and auto- 
matically executed when a link to the internet is available.) 

Still more complex transactions can be realized with the 
use of a remote database indexed by digital watermark fields 
decoded from the poster. For example, the poster may 
promote a concert tour. Fields of the digital watermark may 
identify the artist (by a code or full text), and a web site or 
IP address. The user appliance establishes a link to the 
specified site, and provides the artist identifier. In response, 
the site downloads the tour schedule for that artist, for 
display on the device. Additionally, the downloaded/ 
displayed information can include a telephone number that 
can be used to order tickets or, more directly, can indicate the 
class of seats still available at each (or a selected) venue, and 
solicit a ticket order from the user over the device. The user 
can supply requested information (e.g. mailing address and 
charge card number) over the return channel link (wireless 
or wired, as the case may be), and the ticket(s) will be 
dispatched to the user. In the case of a wireless link, all of 
this can occur while the user is standing in front of the movie 
poster. 

Similar systems can be implemented based on watermark 
data encoded in any other promotional media. Consider 
music videos. Using known TV/computer appliances, water- 
mark data added to such videos can readily be decoded, and 
used to establish links to audio download, CD -sales, fan 
club, concert ticket outlet web sites, etc., as above. 

Even live events offer such watermark-based opportuni- 
ties. The analog audio fed to public address or concert 
speakers can be watermarked (typically before 
amplification) to encode plural -bit digital data therein. A 
next generation personal music appliance (e.g. one with a 
wireless interface to the internet) can include analog record 
capability (e.g. a built-in microphone, analog-to-digital 
converter, MP3 encoder, coupled to the unit's semiconduc- 
tor memory). A user who attends a live event may record an 
excerpt of the music. The watermark can then be decoded, 
and the extracted data used to access the links and commerce 
opportunities reviewed above. 

Cinema movies offer both audio and visual opportunities 
for watermark-based commerce opportunities. Either 
medium can be encoded to convey information of the types 
reviewed above. A personal appliance with image- or audio - 
capture capabilities can capture an excerpt of the audio or 
imagery, decode the watermark data therefrom, and perform 
any of the linking, etc., functions reviewed above. 

The consumer-interest watermarks reviewed above are 
only exemplary. Many others will be recognized as useful. 
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For example, promotional clips presented before a feature 
film presentation can include watermark data that point 
(either by a literally encoded web address link, or by an ID 
code that indexes a literal link in a remote link database) to 
reviewer critiques of the previewed movies. Watermark data 
in a featured film presentation can lead to web sites with 
information about the movie stars, the director, the producer, 
and can list other movies by each of these persons. Other 
watermark-conveyed web links can present opportunities to 
buy the movie on videotape, to purchase the movie 
soundtrack, to buy movie-related toys and games, etc. 
More on Device Control 

Much of the foregoing has focused on watermark encod- 
ing to provide enhanced customer experiences or opportu- 
nities. Naturally, watermarks data can alternatively, or 
additionally, serve the interests of the media owner. 

To illustrate, consider watermarked music. The media 
owner would be best served if the watermark serves dual 
purposes: permissive and restrictive. Permissive ly, music 
appliances can be designed to play (or record) only music 
that includes an embedded watermark signaling that such 
activity is authorized. By this arrangement, if music is 
obtained from an unauthorized source and does not include 
the necessary watermark, the appliance will recognize that it 
does not have permission to use the music, so will refuse 
requests to play (or record). 

As noted, music appliances can respond restrictive ly to 
the embedded watermark data to set limits on use of the 
music. Fields in the watermark can specify any or all of (or 
others in addition to) (a) the types of devices on which the 
music can be played (b) the types of devices on which the 
music can be recorded; (c) the number of times the music 
can be played; (d) the number of times the music can be 
recorded, etc. 

The device restrictions (a) and (b) can be of various types. 
In some embodiments, the restrictions can identify particular 
units (e.g. by serial number, registered owner, etc.) that are 
authorized to play/record the encoded music. Or the restric- 
tions can identify particular classes of units (e.g., battery- 
powered portable players with music memories of less than 
50 megabytes, disk-based dedicated music appliances, gen- 
eral purpose personal computers, etc.) Or the restrictions can 
identify particular performance quality criteria (e.g., two 
channel, 16-bit audio at 44.1 KHz sample rate, or lower 
quality). 

The use restrictions (c) and (d) can likewise be of various 
types. Examples include "do not copy," "copy once only/* 
"unrestricted copying permitted/' "play once," "play N 
times" (where N is a parameter specified elsewhere in the 
watermarked data, or by reference to a database indexed by 
a watermark data field), "unrestricted playing permitted," 
etc. 

It is straightforward to design a music appliance to 
respond to usage limits of zero (e.g. "do not copy") and 
infinity (e.g. "unrestricted copying permitted," and "unre- 
stricted playing permitted"). The device simply examines 
one or more bits in the watermark data, and permits (or 
refuses) an operation based on the value thereof. 

Implementation of the other usage -control restrictions can 
proceed in various ways. Generally speaking, the stored 
music can be altered to give effect to the usage-control 
restrictions. For example, if the music is "record-once/' then 
at the time of recording, the appliance can alter the music in 
a fashion indicating that it now has "do not record" status. 
This alteration can be done, e.g., by changing the watermark 
data embedded in the stored music (or adding watermark 
data), by changing other data stored in association with the 
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music, etc. If the original signal is stored (as opposed, e.g., 
to a streaming signal, such as an internet or wireless 
transmission), it too should be so-altered. 

Likewise with playback limitations. The number of play- 

5 backs remaining can, e.g., be encoded in an updated water- 
mark in the music, be tracked in a separate counter, etc. 

More particularly considering the "copy once" usage 
restriction, an illustrative embodiment provides two distinct 
watermark payload bits:- a "copy once" bit and a "copy 

10 never" bit. When originally distributed (whether by internet, 
wireless, or otherwise), the "copy once" bit is set, and the 
"copy never" bit is un-set. 

When music encoded in this fashion is provided to a 
compliant recording device, the device is authorized to make 

15 one copy. (A compliant device is one that recognizes 
encoded watermark data, and behaves as dictated by the 
watermark.) When this privilege is exercised, the recording 
device must alter the data to ensure that no further copying 
is possible. In the illustrated embodiment, this alteration is 

20 effected by the recording device adding a second watermark 
to both the music, with the "copy never" bit asserted. The 
second watermark must generally be encoded in an 
"orthogonal" domain, so that it will be detectable notwith- 
standing the continued presence of the original watermark. 

25 Compliant equipment must then check for both watermarks, 
and refuse to copy if either is found to have the "copy never" 
bit asserted. 

One advantage to this arrangement Ls that if the water- 
mark signal has undergone some form of corruption (e.g. 

30 scaling or resampling), the first watermark may have been 
weakened. In contrast, the second watermark will be native 
to the corrupted signal, and thus be more easily detected. 
(The corruption may also contribute to the orthogonality of 
one watermark relative to the other, since the two water- 

35 marks may not have precisely the same time base or other 
foundation.) 

An alternative approach is not to encode the "copy never" 
bit in the original music, but leave this bit (in whatever 
manifestation) blank (i.e. neither "1" nor "0"). In transform- 

40 based watermark techniques, this can mean leaving trans- 
form coefficient(s) corresponding to the "copy never" bit 
unchanged. If the watermarking is effected irt the temporal 
sample domain (or spatial domain, for image data), this can 
mean leaving certain samples (pixels) unmodified. The 

45 recording device can then alter the transform coefficients 
and/or samples as necessary to assert the previously- 
unencoded "copy never" bit when the permitted recording is 
made. 

In such a system, compliant recording devices check for 
50 the "copy never" bit in the sole watermark, and refuse to 
make a copy if it is asserted (ignoring the value of any "copy 
once" bit). 

A third approach to "copy once" is to set both the "copy 
once" and "copy never" bits, but set the former bit very 

55 weakly (e.g. using lower gain and/or high frequency DCT 
coefficients that do not survive certain processing). The frail 
"copy once" bit is designed not to survive common 
corruptions, e.g., resampling, scaling, digital to analog 
conversion, etc. To further assure that the "copy once" bit is 

60 lost, the recording device can deliberately add a weak noise 
signal that masks this bit (e.g. by adding a noise signal in the 
frequency band whose DCT coefficient conveys the "copy 
once" bit). In contrast, the "never copy" bit is unchanged and 
reliably detectable. 

65 In such a system, compliant devices check for the "copy 
once" bit in the sole watermark, and refuse to make a copy 
if it is not detected as set. 
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These three examples are but illustrations of many pos- 
sible techniques for changing the rights associated with a 
work. Many other techniques are known. See, e.g., the 
proposals for watermark-based copy control systems for 
digital video at the Copy Protection Technical Working 
Group, http://www.dvcc.com/dhsg/, from which certain of 
the foregoing examples are drawn. See also Bloom et al, 
"Copy Protection for DVD Video," IEEE Proceedings, 
Special Issue on Identification and Protection of Multimedia 
Information, June, 1999. 
Scaleability 

One feature that is desirable in many detector embodi- 
ments is scaleability. This refers to the ability of a detector 
to scale its computational demands to match the computa- 
tional resources available to it. If a detector is running on a 
high performance Pentium III workstation, it should be 
"doing more" than if the same detector is running on a slow 
microcontroller. One way scalability can be achieved is by 
processing more or less chunks of input data (e.g. temporal 
excerpts of music, or blocks/macroblocks of pixels in a 
frame of video data) to decode watermarks. For example, an 
input audio stream might be broken into chunks of one 
second each. A fast processor may complete decoding of 
each chunk in less than a second, permitting it successively 
to process each chunk in the data stream. In contrast, a slow 
processor may require two and a half seconds to decode the 
watermark from a chunk. While it is processing a first chunk, 
the second and third pass by un-decoded. The processor next 
grabs and processes the fourth chunk, permitting the fifth 
and sixth to pass by un-encoded. 

The detector running on the fast processor is clearly more 
difficult to "fool," and yields a decoded watermark of higher 
confidence. But both systems decode the watermark, and 
both operate in "real time." 

The skipping of input data in the temporal (e.g. music or 
video) or spatial (e.g. image or video) domain is but one 
example of how scaleability can be achieved. Many other 
approaches are known to those skilled in the art. Some of 
these alternatives rely on spending more or less time in the 
data analysis phases of watermark decoding, such as cross- 40 
correlation operations. 

Reference has been made to watermarked UIDs as refer- 
ring to a database from which larger data strings (e.g. web 
addresses, musician names, etc.) can be retrieved. In some 
embodiments, the data record referenced by a UID can, in 
turn, point to several other database records. By such 
arrangements, it is often possible to reduce the payload of 
the watermark, since a single UID reference can lead to 
several different data records. 
Production Tools 

In the prior art, the watermark embedded in a source 
material is typically consistent and static through a work — 
unchanging from beginning to end. But as will be recog- 
nized from the foregoing, there are many applications that 
are better served by changing the watermark data dynami- 
cally during the course of the work. According to another 
aspect of the invention, a production tool is provided that 
facilitates the selection and embedding of dynamically- 
changing watermark data. One such embodiment is a soft- 
ware program having a user interface that graphically dis- 
plays the different watermark fields that are being embedded 
in a work, and presents a library of data (textually or by 
icons) that can be inserted into each field, and/or permits the 
user to type in data to be encoded. Another control on the UI 
controls the advance and rewind of the media, permitting the 
user to determine the location at which different watermark 
data begins and ends. Graphical paradigms known from 
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video- and audio -editing tools can be used to indicate the 
starting and ending frames/samples for each different water- 
mark payload. 

Such a tool can be of the standalone variety, or can be 
integrated into the desktop audio- and video -production and 
editing tools offered by vendors such as Avid, Adobe, Jaleo, 
Pinnacle Systems, SoundForge, Sonic Foundry, Xing 
Technology, Prosoniq, and Sonic Desktop Software. 
Payment-based Systems 

Another aspect of the present invention is the use of 
anonymous payment tokens that can be used to obtain 
content on the web. In one embodiment, a token comprises 
a 128-bit pseudo-random number, to which additional bits 
identifying an issuing bank (or other issuing institution) are 
appended. (The additional bits can be the IP address of a web 
server of the bank, a routing number identifying the bank for 
electronic wire transfers, or other identifier.) The 128-bit 
numbers are randomly generated by the bank — commonly 
as needed — and each represents a fixed increment of money, 
e.g. ten cents. 

A consumer wishing to have a store of currency for such 
commerce pays the bank, e.g., 10 in exchange for 100 
tokens. These tokens are transferred electronically to disk or 
other storage in the consumer's computer in response, e.g., 
to a credit card authorization, or may be provided by diskette 
or other storage medium over the counter at a bank branch 
(in which case the consumer thereafter copies the numbers 
into storage of his or her computer). (Outlets other than 
banks can of course be employed for distributing such 
numbers, much in the manner that convenience and many 
grocery stores commonly issue money orders.) 

Imagine that the consumer wishes to view the final quarter 
of a Trailblazer basketball game that aired on television a 
week ago. fThe consumer may have either missed the game, 
or may have seen it but wants to see the last quarter again.) 
The user directs a web browser to a web site maintained for 
such purpose and performs a search to identify the desired 
program. (Typically, the web site is maintained by the 
proprietor that holds the copyright in the material, but this 
need not be the case. Some material may be available at 
several web sites, e.g., maintained by ABC Sports, the 
National Basketball Association, and Sports Illustrated.) The 
search can use any of various known search engines, e.g., 
Infoseek, Verity, etc., and can permit searching by title 
terms, keywords, date of airing, copyright owner, etc. By 
typing in, e.g., the keyword 'Trailblazers' and the date 
'4/26/99/ the consumer is presented a listing of videos 
available for download. One, hopefully, is the requested 
game. With each listing is an indication of an associated 
nominal charge (e.g. 80 cents). 

On clicking on a hypertext link associated with the 
desired basketball game, the viewer is presented a further 
screen with one or more options. The first of the listed 
options is the entire game, with commercials. The charge is 
the nominal charge presented on the, earlier screen (i.e. 80 
cents). Other options may include the first, second, third, and 
fourth quarters of the game individually, each of which — 
save the last, costs 20 cents. The last may be charged at a 
premium rate, e.g., 30 cents. Clicking on the desired video 
option yields a further screen through which payment is 
effected. 

To pay for the requested video, the consumer instructs his 
or her computer to transfer three of the earlier-purchased 
tokens over the web to the video provider. Various user 
interface metaphors can be employed to facilitate this 
transfer, e.g., permitting the user to type the amount of 
money to be transferred in a dialog box presented on-screen, 
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or dropping/dragging icons representing tokens from an 
on-screen "wallet 1 ' to an on-screen "ticket booth" (or over an 
icon or thumbnail representing the desired content), clicking 
on an "increment" counter displayed adjacent the listing of 
the content, etc. Once the consumer has authorized a transfer s 
of sufficient tokens, the consumer's computer sends to the 
web site (or to such other web address as HTML encoding 
in the viewed web page may indicate) the tokens. This 
transmission simply takes the form of the three 128+bit 
numbers (the indicating the bank identifier) — in what- 10 
ever packet or other format may be used by the internet link. 
Once dispatched in this manner, the tokens are deleted from 
the user's computer, or simply marked as spent. (Of course, 
in other embodiments, a record of the expenditure may be 
stored in the consumer's computer, e.g., with the token 15 
contents and a record of the audio or video purchase to 
which they were applied.) 

Since the amount of money is nominal, no encryption is 
provided in this embodiment, although encryption can natu- 
rally be provided in other embodiments (e.g., either in 20 
sending the tokens from the user to the web site, or earlier, 
in sending the tokens to the user). As will be seen, provided 
that the media provider immediately sends the tokens to the 
bank in real time, encryption is a nice feature but not 
mandatory 25 

On receipt of the token data, the web site immediately 
routes the token data to the identified bank, together with an 
identifier of the media provider or account to which the 
funds represented thereby are to be credited. The bank 
checks whether the 128-bit numbers have been issued by 30 
that bank, and whether they have already been spent. If the 
numbers are valid, the bank updates its disk-based records to 
indicate that the three tokens have been spent and that the 
bank now owes the media supplier 30 cents, which it may 
either pay immediately (e.g., by crediting to an account 35 
identified by the media provider) or as one lump sum at the 
end of the month. The bank then sends a message to the web 
site confirming that the tokens were valid and credited to the 
requested account, (Optionally, a message can be sent to the 
purchaser of the tokens (if known), reporting that the tokens 40 
have been redeemed.) 

In response, the web site begins delivery of the requested 
video to the consumer. In the illustrated embodiment, the 
video is watermarked prior to delivery, but otherwise sent in 
unencrypted fashion, typically in streaming format, but 45 
optionally in file format. (Encryption can be used in other 
embodiments.) The watermarking in the illustrated embodi- 
ment is accomplished on-the-fly and can include various 
data, including the date of downloading, the download site, 
the destination IP address, the identity of the purchaser (if 50 
known), etc. 

The large size of the video and the small charge assessed 
therefor provide disincentives for the consumer making 
illicit copies. (Especially as to archival material whose value 
decays with time, there is not much after-market demand 55 
that could be served by illicit copies, making third party 
compilation of such material for re -distribution financially 
unattractive. First run video, and material that keeps a high 
value over time, would not be as well suited for such 
distribution, and could better employ technology disclosed 60 
elsewhere herein.) 

In some embodiments, the integrity of the received video 
is checked on receipt. This feature is described below in the 
section entitled Watermark-Based Receipts. 

In the illustrative system, nothing in the tokens indicates 65 
the identity of the purchaser. The web site knows the IP 
address of the site to which video was delivered, but need 
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not otherwise know the identity of the purchaser. The bank 
would probably maintain a record of who purchased the 
tokens, but need not. In any event, such tokens could 
thereafter be exchanged among consumers, resulting in 
anonymity from the bank, if desired. 

As described above, the video excerpts from which the 
consumer can select include commercials. At some sites, 
video may be provided from which the commercials have 
been excised, or which is delivered in a manner that skips 
past the commercials without transmitting same to the 
consumer. Such video will naturally command a premium 
price. In some embodiments, the difference in price is 
electronically credited as compensation to accounts main- 
tained for (or by) the advertisers, whose advertisements are 
not being viewed by such consumers. (The identification of 
advertisers to be credited is desirably permanently encoded 
in the video, either throughout the video (if the video has had 
the commercials removed therefrom), or by data in the 
commercials themselves (which commercials are skipped 
for transmission to the consumer, but can still be decoded at 
the video head-end. Such encoding can be by in -band 
watermarking or otherwise.) 

While the foregoing discussion particularly considered 
video as the desired content, the same principles are equally 
applicable in connection with audio, still imagery, and other 
content. 

The token-based payment method is but one of many that 
can be employed; the literature relating to on-line payment 
mechanisms is extensive, and all such systems can generally 
be here-employed. 

Tracking 128-bit tokens can be a logistical problem for 
the bank. One approach is to have a memory with 10 128 
locations, and at each location store a two-bit value (e.g. 
00=never issued; 01=issued but not spent; 10=issued and 
spent; ll=reserved). More complete data could alternatively 
be stored, but such a memory would be unpractically large. 

One alternative approach is to hash each 128-bit number, 
when issued, to a much smaller key value (e.g. 20 bits). A 
memory with 10 20 locations can be indexed by this key. 
Each such location can include four data: an issued 128-bit 
token number that hashes to that value, first and second date 
fields indicating the date/time on which that token was 
issued and redeemed, respectively, and a link specifying the 
address of a next memory location. That next memory 
location (outside of the original 10 20 locations) can include 
four more data, this time for a second issued-128-bit token 
number that hashed to the original key value, two date fields, 
and again with a link to a subsequent storage location, etc. 

When a 128-bit random number is generated, the original 
memory location indexed by the hash code of that number 
is checked for an earlier number of the identical value (to 
avoid issuance of duplicate tokens). Each successive loca- 
tion in the linked chain of memory locations is checked for 
the same 128-bit number. When the end of the linked chain 
is reached, the bank knows that the 128-bit random number 
has not previously been issued, and writes that number in the 
last-addressed location, together with the date of issuance, 
and a link to a next storage location. 

When a 128-bit token is received, the same linked-list 
processing occurs to identify a first location, and to there- 
after step through each subsequent location until a match is 
found between the token number and the number stored in 
one of the linked memory locations. When found, that 
number is marked as redeemed by writing a redemption 
date/time in the corresponding field. If the search reaches the 
end of the linked chain without finding a match between the 
stored numbers and the token number, the token is treated as 
invalid (i.e. not issued by that bank). 
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Other manners of tracking the large number of possible 
token numbers can of course be used; the foregoing is just 
exemplary. Or the tokens needn't be tracked at all. Such an 
arrangement is highly practical if the token has sufficient 
bits. With the illustrated 128 bits, for example, the chance of 
two identical tokens being issued is infinitesimally small, so 
checking for duplicate issuance can be omitted if desired. In 
such case, the bank can simply maintain an ordered list of 
the token numbers still outstanding and valid. As new tokens 
are dispensed, their token numbers are added to the list. As 
tokens are redeemed, their numbers are deleted from the list. 
Known list processing techniques can be employed to speed 
such search, update, and delete actions. 
Watermark-based Receipts 

Pay-for-content applications commonly assume that if 
content is transmitted from a server (or head-end, etc.), it is 
necessarily received. Sometimes this assumption is wrong. 
Network outages and interruptions and internet traffic load 
can diminish (e.g., dropped video frames), or even negate 
(e.g., failed delivery), expected consumer enjoyment of 
content. In such cases, the consumer is left to haggle with the 
content provider in order to obtain an adjustment, or refund, 
of assessed charges. 

Watermarks provide a mechanism for confirming receipt 
of content. If a watermark is detected continuously during a 
download or other delivery event, a software program (or 
hardware device) can issue an electronic receipt attesting 
that the content was properly delivered. This receipt can be 
stored, and/or sent to the content distributor to confirm 
delivery. 

In one embodiment, a content receiving device (e.g., 
computer, television or set-top box, audio appliance, etc.) 
periodically decodes a watermark from the received content 
to confirm its continued reception. For example, every five 
seconds a watermark detector can decode the watermark and 
make a record of the decoded data (or simply record the fact 
of continued detection of the same watermark). When a 
changed watermark is detected (i.e., reception of a different 
content object begins), the duration of the previously- 
received content is logged, and a receipt is issued. 

In a related embodiment, the last portion (e.g., 5 seconds, 
frame, etc.) of the content bears a different "end of content" 
watermark that triggers issuance of a receipt. Such a water- 
mark can indicate the length of the content, to serve as a 
cross-check against the periodic watermark polling. (E.g., if 
periodic sampling at 2 second intervals yields 545 samples 
corresponding to the same content, and if the "end of 
content" watermark indicates that the content was 1090 
seconds long, then receipt of the entire content can be 
confirmed.) 

In another embodiment, the watermark can change during 
the course of the content by including, e.g., a datum that 
increments every frame or other increment of time (e.g., 
frame number, time stamp, etc.). A watermark detector can 
monitor the continued incrementing of this datum through- 
out the content to confirm that no part was garbled (which 
would destroy the watermark) or was otherwise missing. 
Again, at the end of delivery, the receiving system can issue 
a confirmation that XXX frames/seconds/etc. of the identi- 
fied content were received. 

One application of such technology is to bill for content 
based on receipt, rather than transmission. Moreover, bill- 
ings can be adjusted based on percentage of content-value 
received. If delivery is interrupted mid-way through (e.g., by 
the consumer disabling the content-receiving device), the 
nominal billing for the content can be halved. Some pro- 
longed content, e.g., televised/web-broadcast university 
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classes, cannot be "consumed" in one session, and are thus 
particularly well suited for such pay-as-you-consume bill- 
ing. 

Another application of such technology is in advertising 

5 verification. Presently, ads are tracked by transmission or, 
less frequently, by detection of an embedded code on receipt 
(c.f., Nielsen Media Research's U.S. Pat. Nos. 5,850,249 
and 5,737,025). However, such reception-detectors — once 
triggered — generally do not further note the length of time 

10 that the advertising was received, so the same data is 
produced regardless of whether only five or fifty seconds of 
a commercial is presented. Watermark monitoring as con- 
templated herein allows the duration of the advertising 
impression to be precisely tracked. 

15 In one application of this technology, recipients of adver- 
tising are provided incentives for viewing advertising in its 
entirety. For example, a content-receiving device can 
include a watermark detector that issues a receipt for each 
advertisement that is heard/viewed in its entirety. These 

20 receipts can be redeemed, e.g., for content tokens as 
described elsewhere herein, for monetary value, etc. In some 
embodiments, receipts are generic and can all be applied to 
a desired premium, regardless of the advertisements through 
which they were earned. In other embodiments, the receipts 

25 are associated with the particular advertisers (or class of 
advertisers). Thus, a TV viewer who accumulates 50 receipts 
from advertising originating from Procter & Gamble may be 
able to redeem same for a coupon good for $2.50 off any 
Procter & Gamble product, or receipts from Delta Airlines 

30 may be redeemed for Delta frequency flier miles (e.g., at a 
rate of one mile per minute of advertising). Such incentives 
are particularly useful in new forms of media that give the 
consumer enhanced opportunities to fast-forward or other- 
wise skip advertising. 

35 (Although the foregoing "receipt" concept has been 
described in conjunction with watermark data (and use of 
watermark technology is believed to be inherently advanta- 
geous in this application), the same principles can likewise 
be implemented with ancillary data conveyed by other 

40 means.) 

Master Global Address 

As suggested above, it is desirable that each piece of 
content have a web address (the "Master Global Address" 
(MGA), or "Master IP Address") associated with it. Such 

45 address is typically conveyed with the content, e.g., by an IP 
address watermarked therein. 

Consider a consumer who downloads a streaming video 
having an English language soundtrack. The viewer may not 
speak English, or may otherwise prefer to listen to the 

50 soundtrack in another language. The user can decode the 
watermark data embedded in the video and initiate a link to 
the associated web address. There the user is presented with 
a list of soundtracks for that content object in other lan- 
guages. The viewer can click on the desired language and 

55 receive same via a second simultaneous transmission (e.g., 
a second socket channel). The consumer's audio/video 
appliance can substitute the desired audio track for the 
default English track. 

If the streaming video and the alternative soundtrack are 

60 hosted on the same server, synchronization is straightfor- 
ward. The process governing transmission of the alternative 
soundtrack identifies the process that is streaming video to 
the same IP address. Based on SMPTE, or other time/frame 
data, the former process syncs to the latter. (If the two data 

65 streams don't originate through the same server, time/frame 
data can be relayed as necessary to the alternative 
soundtrack server to effect synchronization.) 
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Another application of the Master Global Address is to TCP/IP or HTML instructions) authorizing an application on 

serve as a point to which monitoring stations can report the the user's computer to open the file in the requested manner, 

presence, or passage, of content. Consider, for example, a (The default application for JPG applications can then 

copyright-aware node through which content signals pass, automatically be launched, or the computer may first query 

e.g., a computer node on a network, a satellite transponder, 5 the user whether another application should be used 

etc. Whenever the node detects passage of a media object instead.) 

(e.g., by reference to a file extension, such as MP3, JPG, Clicking on option (d) proceeds as above, and permits full 
AVI, etc.), it sends a "ping" over the internet to the address use of the image on the computer. Moreover, the MGA site 
encoded in the object, simply reporting passage of the sends a digital certificate to the user's computer memorial- 
object. Similar monitoring facilities can be provided in end 10 izing the usage rights purchased by the consumer, 
user computers, e.g., reporting FileOpen, FileSave, Printing, In this particular arrangement, no access control is placed 
or other use of content bearing MGA data. on the content, e.g., by encryption, secure container 
This system can be expanded to include "ping" and technology, or the like. The nominal fees, and the ease of 
"pong" phases of operation. When a software application (or licensing, make it simple for the user to "do the right thing" 
a user appliance, such as a video or audio playback device) 15 and avoid copyright liability. In other embodiments, of 
encounters a media object (e.g., at time of file open, at time course, known access control techniques can be used to limit 
of playback, etc.), it pings the MGA site to report the use of the object until the requisite payment has been made, 
encounter. The MGA site "pongs" back, responding with Naturally, records of all such transactions are also logged 
instructions appropriate to the encounter. For example, if the at the MGA site. 

object requires payment of a fee before full functionality or 20 Clicking on option (e) opens a browser window on the 

access is to be granted, the MGA site can respond to the user's computer to a web site that presents a complete listing 

application with instructions that the object be used (e.g., of license options available for that image. (The address of 

played back) only in some crippled state preventing the this web site is included in customization data relayed to the 

user's full enjoyment (e.g., impaired resolution, or impaired user device from the MGA site, but not explicitly shown to 

sound quality, or excerpts only, etc.). The MGA site can also 25 the user on the menu.) Through such web site, the user can 

inform the user application of the terms (e.g., payment) by select desired rights, effect payment, and receive the neces- 

which full functionality can be obtained. The application can sary authorization for software applications on the user's 

graphically or audibly present such information to the user, computer (or other media appliance) to open and/or process 

who can authorize a payment, if desired, so that the content the content. 

can be enjoyed in a less- (or un-) crippled state. On receipt 30 The object on which the user "clicks" needn't be an icon, 

of the payment authorization, the MGA site can inform the It can be an image or other graphical representation. (And a 

user application that enhanced access/usage rights have been "click" isn't necessary; a voice command or other signal 

purchased, and that the application may proceed accord- may be used to the same effect with an audio clip or 

ingly. selection.) 

Yet another application of the MGA is to present the user 35 Consider the popular merchandising of books and CDs 

of a content object a menu of options that is customized to over the internet. A JPG or other image file depicting the 

that object. cover of a book, or the artwork of a CD cover, can be treated 

In current graphical operating systems, when a user clicks as a media object, and can include a watermarked MGA 
on an icon (e.g., with the right mouse button), a menu is pointer. Right-clicking on such an image of a book cover 
presented detailing actions that can be undertaken in con- 40 could, through the MGA site, present to the user a menu of 
nection with the icon, or the file represented thereby. Such options that includes — in addition to those normally pre- 
options are pre-programmed (i.e., static), and are typically sented in conjunction with a JPG file — the following: 
determined by the operating system based solely on the file ( a ) "See the review of this book published in the New 
extension. York Times on Apr. 19, 1999" 

In accordance with this aspect of the present invention, 45 (5) « See me Kst of reviews of this book at Amazon.com" 

clicking on an icon representing a media object initiates an (c) ^ nter ycmr own rev iew of this book, for posting on 

internet link to the MGA site associated with the object. The Amazon com" 

MGA site responds with data that is used to customize the /JX A ' , , , c iL . , t 4 A „ 

c r *u • * (d) See today s sales rank of this book at Amazon.com 

menu of options presented to the user in connection with that ; ( (m , . . . , , + r 

particular object. 50 ( e ) Purchase tms book from Amazon.com for $16.95 

Consider an icon representing a JPG image file. Right- (0 "Purchase this book from Bamesandnoble.com for 

clicking on the icon may yield a menu that gives the user $ 19 - 95 ™ d receive a $ 5 -°° credit towards y° ur ncxt 

various options presented by the operating system (e.g., purchase" 

delete, compress, rename), and additional options custom- (g) " Link t0 tDe weD site that tells about the release of this 

ized in accordance with data from the object's MGA site. 55 title as a motion picture (presently scheduled to open on 

These customized options may include, e.g., O ct - 10, 1999)" 

(a) open in 100x150 pixel format for free; ( h ) " u *k t0 ^ Yaho ° lis ung of web silcs relating to this 

(b) open in 480x640 pixel format for ten cents; t „ . 

, v . ■ * n (0 Search Lycos for listings relating to this book. 

(c) open in 960x1280 pixel format for twenty cents; 6Q If ^ ^ Qnc of ^ purcW options frQm |fac 

(d) purchase rights to use this image in a newsletter menu, a pre-stored e-commerce profile— containing the user 
having a circulation of under 1000 for $1.25; name , credit card number, billing address, ship-to address, 

(e) display a complete listing of license options. etc., possibly in the form of an encrypted object — could be 
Clicking on options (b) or (c) initiates a commerce sent to the MGA site (or to the bookseller) to effect the 

application through which funds are electronically trans- 65 purchase, or such selection could initiate display of addi- 

ferred to the MGA site (by the above-described tokens or tional screens or sub -menus through which the user would 

otherwise). In response, the MGA site responds (e.g., with manually enter or select such information for transmission. 
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Others of the selections cause a new browser window to specified URL, the client computer may direct such links to 

open on the user's computer, opening to a URL specified in a default name server address instead (stored locally or 

data relayed from the MGAsite but not displayed to the user remotely). If that server doesn't recognize the object ID, it 

in the menu. Appropriate HTML instructions can be gener- can return an error code, or pass the query on to other name 

ated to effect a particular query or other operation at the 5 servers. Those servers, in turn, can pass the query along to 

specified URL, still other name servers if they don't recognize the object ID. 

In some embodiments, the customized menu presents In this fashion, an exponentially-large number of name 

only a single choice in addition to those normally provided servers might be quickly polled for information relating to 

by the operating system, e.g., "Link to home." Clicking on the identified object. Alternatively, rather than encoding the 

this option opens a browser window to a home page at the 10 complete IP address of the name server in an object 

MGA for that object. On that page, the user is presented with watermark, the first N (e.g., 16) bits of the object ID might 

all of the foregoing options, and more (possibly including be used as a short-hand for one of 65,536 predetermined 

advertising graphics or multi-media). Such objects can serve name server addresses, in accordance with data stored 

as powerful marketing agents. Returning to the example locally (e.g., on RAM or disk in the user's computer) or 

discussed above, a JPG image file of a book cover may have, 15 remotely (e.g., at a default name server IP address), 

as its MGA, a web page hosted by a particular bookseller, While the basic concept idea behind embedding MGA 

providing purchase options and other information for that data within an object is to point to a repository of data about 

book. Marketing of books (or CDs, or cars, or consumer the object, a pointer the other way may be achieved as well, 

appliances, or virtually anything else) can be effected by As noted, the "ping" application of MGA data permits an 

disseminating such vendor-issued JPGs as widely as pos- 20 MGA site to be informed of sites through which its object 

sible. Some book cover JPGs may be distributed by passes. More generally, the MGAsite can log the originating 

Amazon.com, others by Barnes&Noble.com, others by address of each query it receives. Each such address can be 

Borders.com — each pointing back to a different MGA presumed to have (or have had) a copy of the corresponding 

through which purchase transactions for that book may be object. Media owners can thereby track the dissemination of 

performed. 25 copies of their media objects — at least insofar as use of such 

Returning to the MGA-customized menus, these needn't objects entails communicating with the associated MGA 

be limited to menus resulting from clicking on an icon or site. 

image (or signaling during an audio excerpt). Drop-down Such tracking offers a great number of opportunities, 

menus in application programs can likewise be populated some in the area of commerce. The MGAsite corresponding 

with customized options, in accordance with customization 30 to the cover art of a Garth Brooks CD, for example, can 

data obtained from the MGA site for the object presently provide a listing of IP addresses of persons interested in that 

being accessed or used. Most graphical operating systems CD. Email or promotional data objects (e.g., audio clips) can 

and application programs have well developed toolsets be sent to that list of addresses when a subsequent Garth 

permitting such menu customization. Again, other data Brooks CD is released. 

relayed from the MGA site is not shown to the user, but is 35 Such tracking also opens up a new dimension of internet 

employed by the computer (e.g., a browser program) to carry searching. Presently, internet search engines use a brute 

out menu options selected by the user. force approach, visiting millions of pages across the web in 

Again the foregoing techniques are equally applicable for order to identify, for example, a dozen instances of a given 

still images, audio, video, and other forms of content, and photograph file, MGAs offer a shortcut to such brute force 

can readily be adapted for use both with general purpose 40 approaches. With the present technology, a search engine 

computers, software applications, and specialized media can find a single instance of a photograph file and, by 

appliances. detection of the MGA data watermarked therein, link to the 

While, for expository convenience, the foregoing discus- corresponding MGA site. From the MGA site, the search 

sion contemplated embedding a literal URL address in the engine can obtain a listing (if such queries are authorized) of 

object as the MGA, more typically this is not the case. 45 some or all of the other sites known by the MGA site to have 

Instead, the MGA more commonly comprises identification copies of that photograph file. (Providing such data to search 

data for the object (e.g. a 128-bit random ID), together with engines is a commerce opportunity for such MGA sites, 

the URL for a name server computer that serves many which may permit such access to its listing of sites only in 

(perhaps millions) of such objects (an example of the latter exchange for a fee. Or the MGA site may arrange to collect 

is the Digimare MarcCentre server). 50 a tribute payment from the search engine proprietor each 

To obtain the desired data as detailed above, the user's time the engine responds to a user query using data collected 

computer (sometimes termed a client computer) links to the from the MGA site.) 

name server computer and provides the ID of the object Many of the addresses logged by the MGA may not be 

being processed. The name server computer uses this ID to publicly-accessible data stores. The search engine can check 

query a database, and obtains from the database the current 55 each listed address to ensure that the desired object is present 

IP address to which such queries should be routed. The name and accessible before adding the address to its database, 

server computer can relay the request from the client com- Covert Tracing 

puter to the correct destination address, or can return the Co-pending application Ser. No. 09/185,380 describes 

correct destination address to the client computer, which can anti-counterfeiting technology that looks for the presence of 

initiate such a link itself. By such arrangement, the IP 60 digital data corresponding to bank note imagery in a com- 

address ultimately associated with an object can be easily puter system, and makes a covert record of any attempt to 

changed as needed, simply by changing the corresponding process such data (e.g., Scan, FileOpen, FileSave, Print, 

record in the name server database, without rendering obso- Edit, etc.). Such records are hidden from the user of the 

lete legacy objects having out-of-date addresses encoded system (using, e.g., various data encryption and obscuring 

therein. 65 techniques), but authorized law enforcement officials are 

In some embodiments, the URL of the name server provided tools by which these records can be recovered. The 

needn't be included in the watermark. In the absence of a forensic data thereby obtained may prove useful in pros- 
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ecu ting counterfeiters. (Knowledge that a computer may be as needed. Data objects of disparate formats thus can readily 

covertly storing evidence of attempted counterfeiting be handled within a single, simple application program, 

actions may prove as, or more, valuable in deterring coun- since the program needn't concern itself with the varying 

terfeiting than the covert records themselves.) formats for the associated meta-data (assuming the name 

The same techniques can be employed to deter unautho- s servers provide this data in standardized format). Substantial 

rized processing of audio, image, video, or content by media flexibility in programming and object formatting is thereby 

pirates. In one embodiment, a computer's operating system achieved. 

(including peripheral device drivers) monitors various data Returning to the internet search engine example described 

within the system (e.g., data sent to writeable storage media, above > MGAi > ma y become recognized as repositories rich 

or sent via a serial port or network connection, etc.) for data 10 m meta-data for media objects. Specialized search engines 

bearing a do-not-copy watermark. The presence of such data ™? focus their data collection around such sites, and be 

being sent, e.g., to a writeable disk or to a remote computer, able t0 > lden ^ , the ? f ** corresponding to 

. . . .. , , . various boolean combinations of meta-tag parameters, 

indicates that the do-not-copy instruction has been circum- ^ Management/C oatainers 

vented. In such case, the operating system writes one or Much has been wriUen Qn the t ic of assct d ht& 

more covert records memorializing the activity, for possible is management. Sample patent documents include U.S. Pat. 

use in criminal prosecution if the computer is lawfully Nos. 5,892,900, 5,715,403, 5,638,443, 5,634,012, 5,629,980 

seized- and laid-open European application EP 862,318. Much of 

The example just-provided is but one of many monitoring me technical work is memorialized in journal articles, which 

and response techniques that may be employed to deter can be identified by searching for relevant company names 

circumvention of copy-protection or other access control 20 and trademarks such as IBM's Crypto lope system, Portland 

systems. Generally speaking, if content data is found where Software's ZipLock system, the Rights Exchange service by 

it shouldn't be, or is found used as it shouldn't be used, a Softbank Net Solutions, and the DigiBox system from 

corresponding record should be made. (Other intervention InterTrust Technologies. 

actions can be triggered as well; covert tracing is desirably An exemplary asset management system makes content 

just one of several parallel responses to suspected hacking.) 25 available (e.g. from a web server, or on a new computer's 

Meta-data Accessed Using Watermarks hard disk) in encrypted form. Associated with the encrypted 

Meta-data, in formats known as XML, SGML, and content is data identifying the content (e.g. a preview) and 

HTML, is widely used to communicate information about data specifying various rights associated with the content. If 

digital objects (e.g., author, keywords, price, rights, caption, a user wants to make fuller use of the content, the user 

etc.). More generally, meta-data can be thought of as any 30 provides a charge authorization (e.g. a credit card) to the 

data construct which associates the name of a property (e.g., distributor, who then provides a decryption key, allowing 

"author"), with the value of the property (e.g., "Mark access to the content. (Such systems are often realized using 

Twain"). Such data commonly appears in a tag format, such object-based technology. In such systems, the content is 

as the following: commonly said to be distributed in a "secure container.") 

<META NAME=" author" CONTENT="Mark Twain"> 35 Desirably, the content should be marked (personalized/ 

Meta-data is commonly exchanged between server and serialized) so that any illicit use of the content (after 

client computers in conjunction with the digital objects to decryption) can be tracked. This marking can be performed 

which they relate (e.g., the text of a Mark Twain book). with watermarking, which assures that the mark travels with 

As detailed herein, an important application of water- the content wherever — and in whatever form — it may go. 

marking is likewise to convey information about media — in 40 The watermarking can be effected by the distributor — prior 

this case embedded within the media content itself (e.g., to dissemination of the encrypted object — such as by encod- 

providing unique identification, establishing some basic ing a UID that is associated in a database with that particular 

behaviors such as do not copy, and providing links to container. When access rights are granted to that container, 

extended functionality). • the database record can be updated to reflect the purchaser, 

For meta-data to be useful, it must be linked to associated 45 the purchase date, the rights granted, etc. An alternative is to 

content, whether in the context of a browser, application include a watermark encoder in the software tool used to 

program, operating system, asset management system, access (e.g. decrypt) the content. Such an encoder can 

search engine, etc. However, as detailed below, the content embed watermark data in the content as it is released from 

and the associated meta-tags needn't always be conveyed the secure container, before it is provided to the user. The 

together. 50 embedded data can include a UID. This UID can be assigned 

Consider an application program or other client process by the distributor prior to disseminating the container, 

that receives a watermarked media object. The watermark Alternatively, the UID can be a data string not known or 

includes an MGA for that object (which, as noted above, created until access rights have been granted. In addition to 

may not specify an ultimate IP address). Stored at the MGA the UID, the watermark can include other data not known to 

site is meta-data corresponding to the object. By linking to 55 the distributor, e.g. information specific to the time(s) and 

the MGA site identified by the object's watermark, the client manner(s) of accessing the content, 

computer can obtain the meta-data corresponding to the As noted earlier, access rights systems can be realized 

object. This data can be stored at the client computer and with watermarks without containers etc. For example, in a 

used just as any other meta-data, e.g., to define the local trusting world, copyrighted works can be freely available on 

functions that should be available for use with that object 60 the web. If a user wishes to make lawful use of the work, the 

(e.g., buy, search, etc.) user can decode its watermark to determine the work's terms 

A particular example is an on-line catalog of stock pho- and conditions of use. This may entail linking to a web site 

tography. Each photograph is watermarked with MGA data. specified by the embedded watermark (directly, or through 

To identify the photographer, copyright date, price, tele- an intermediate database), which specifies the desired infor- 

phone number, subject, etc., an application program can link 65 mation. The user can then arrange the necessary payment, 

to the MGA site for that photograph, and obtain the corre- and use the item knowing that the necessary rights have been 

sponding meta-data. This data can then be displayed or used secured. 



11/25/2003, EAST Version: 1.4.1 



US 6,4' 

27 

Remote Reconfiguration of Watermark Detectors 

In some cases, it is desirable to reconfigure watermark 
detectors remotely. Such functionality is desirable, for 
example, if a watermark system is hacked or otherwise 
compromised. 

In accordance with this aspect of the present invention, 
some aspect of a watermark detector's operation is changed 
in response to a command. The change can take various 
forms. In watermark systems employing pseudo-random key 
data (e.g., spread spectrum spreading signals), the pseudo- 
random signal used for detection can be changed. In systems 
using DFT processing, the mapping between message bits 
and DFT coefficients can be changed. In still other systems, 
the decoding can proceed as before, but the significance of 
one or more bits can be changed (e.g., bits that were 
normally interpreted as defining Field A can be interpreted 
as defining Field B, and vice versa). In yet other systems, the 
decoding can proceed as before, but the response of a device 
to a given watermark signal can be changed. In still other 
systems, a set of software instructions can be re- written or 
re-ordered to effect a change in detector operation. 

The command can be conveyed in various ways. In one 
embodiment, it can be a trigger bit in the watermark pay load. 
Normally the bit has a value of "0." If the bit has a value of 
"1," the detector system responds by changing its operation. 
A trigger pattern can also be established, so that detection of 
a certain combination of bits in the watermark payload 
serves to trigger the change. Reserved states of certain data 
fields are examples of patterns that might be employed. 

The command can also be conveyed through another 
channel different than the watermark channel (e.g., an SCA 
channel of an FM broadcast, or the sub-titling data channel 
of video broadcasts, or header data within an MPEG data 
stream, etc., etc.). 

The change can proceed in accordance with a pre- 
programmed rule (e.g., codes progressing successively 
through a numerically or algorithmically-determined 
progression), or the change can proceed in accordance with 
data specified elsewhere in the payload of the watermark 
bearing the trigger bit (e.g., instead of being interpreted in 
normal fashion, the non-trigger bits of the detected water- 
mark can define a new pseudo -random key data. Or the 
change can proceed in accordance with data conveyed in 
successively-presented watermark payloads, as might be 
done in video encoding where each frame of video can 
convey further watermark information. (This latter arrange- 
ment is one offering a high-bandwidth re-programming 
channel through which, e.g., extensive firmware instructions 
might be transferred to the detector to replace instructions 
earlier stored.) 

By such arrangements, greatly increased detector versa- 
tility and functionality can be achieved. 
Conclusion 

Many diverse embodiments are reviewed above — each 
with a unique set of features. (Still others are disclosed in the 
assignee's patents incorporated by reference.) To provide a 
comprehensive disclosure without unduly lengthening the 
specification, applicants incorporate by reference the patents 
and patent applications referenced above. 

This specification should be construed as explicitly teach- 
ing that features illustrated in one such embodiment can 
generally be used in other embodiments as well. Thus, for 
example, a date field was not particularly discussed in 
connection with payload data for video watermarking. Nor 
were "play once" watermarks so -considered. The inclusion 
of a calibration signal with (or as part of the watermark is 
shown in embodiments of the issued patents, but is not 
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belabored in the above-described embodiments. Likewise 
with "simple universal codes." The pre-stored commerce 
profile described in one of the foregoing embodiments is 
equally applicable to other embodiments as well. Likewise, 

5 the presentation of advertising was discussed in connection 
with one embodiment but not others, although it, too, is 
generally applicable. All of these concepts are familiar at 
Digimarc and are regarded as generally applicable through- 
out the work expressed in Digim arc's patent disclosures. 

10 Practicality prevents an exhaustive recitation of each indi- 
vidual permutation and combination. 

Having described and illustrated the principles of our 
invention with reference to illustrative embodiments, it will 
be apparent that the detailed arrangements can be modified 

15 in arrangement and detail without departing from such 
principles. 

For example, while reference has been made to various 
uses of wireless, it should be understood that such reference 
does not just cover FM broadcast, and wireless internet 

20 networking and the like, but also includes other wireless 
mechanisms. Examples include cell phones and direct sat- 
ellite broadcast. 

Likewise, while certain embodiments were illustrated 
with a watermark payload of 100+ bits, in other systems 

25 much smaller (or sometimes larger) payloads are 
desirable — sometimes as small as 1-8 bits. 

While the foregoing examples have each been illustrated 
with reference to a particular media type (e.g., video, audio, 
etc.), it will be recognized that the principles of each 

30 embodiment find application with the other media types as 
well. 

Certain of the appliances contemplated above require user 
interfaces more sophisticated than are presently typical on 
such devices. The simplicity of the underlying audio appli- 

35 ance can be preserved, in many instances, by using a 
palmtop computer — coupled by infrared or otherwise — as a 
temporary user interface to the appliance. Some of the 
processing capability can likewise be off-loaded to an ancil- 
lary palmtop. (Palmtop is here meant to refer generally to 

40 any pocket-size programmable computing device.) 

Unless otherwise stated, it should be understood that the 
digital music, video, and imagery contemplated herein is not 
of any particular form or format. Audio, for example, can be 
of various forms, both streaming and non-streaming, and of 

45 various formats (e.g. MP3, MP4, MS Audio, Windows 
Media Technologies, RealAudio, *.WAV, MIDI, Csound, 
Dolby's Advanced Audio Codec (AAC), etc. 

Having described and illustrated the principles of the 
invention with reference to illustrative embodiments, it 

50 should be recognized that the invention is not so limited. 
For example, while digital watermarking typically does 
not leave any human-apparent evidence of alteration or data 
representation, certain of the foregoing applications do not 
require this. Hie markings used may be visible, and even 

55 conspicuous, without impairing essential functionality. 
Thus, bar codes, data glyphs, OCR markings, and other 
machine-readable indicia may be substituted, depending on 
the particular application requirements. 
While the detailed embodiments were generally described 

60 with reference to desktop computers, it is recognized that 
such devices will increasingly be supplanted by other digital 
appliances, including general purpose personal digital 
assistants, multifunction cell phones, and specialized 
devices. Moreover, the power and utility of the above- 

65 detailed embodiments and devices can be further enhanced 
by employing various wireless communications 
technologies, including the evolving Bluetooth standard. 
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The implementation of the watermark encoding and 
decoding systems is straightforward to artisans in the field, 
and thus not belabored here. Conventionally, such technol- 
ogy is implemented by suitable software, stored in long term 
memory (e.g., disk, ROM, etc.), and transferred to tempo- 5 
rary memory (e.g., RAM) for execution on an associated 
CPU. In other implementations, the functionality can be 
achieved by dedicated hardware, or by a combination of 
hardware and software. Reprogrammable logic, including 
FPGAs, can advantageously be employed in certain imple- 
mentations. 

It should be recognized that the particular combinations of 
elements and features in the above-detailed embodiments 
are exemplary only; the interchanging and substitution of 
these teachings with other teachings in this and the 15 
incorporated-by-reference patents/applications are also con- 
templated. 

In view of the wide variety of embodiments to which the 
principles and features discussed above can be applied, it 
should be apparent that the detailed embodiments are illus- 20 
trative only and should not be taken as limiting the scope of 
the invention. Rather, we claim as our invention all such 
modifications as may come within the scope and spirit of the 
following claims and equivalents thereof. 

We claim: 25 

1. A method for controlling operation of a device using a 
watermark comprising: 

decoding a watermark embedded in a signal of a media 
type, where the watermark includes a command signal 
used to trigger a change in operation of the watermark 30 
detector; 

based on the command signal, changing operation of the 
watermark detector, including changing how the water- 
mark detector decodes or interprets a watermark in a 
signal of the media type; 35 

after changing operation of the watermark detector, 
decoding a usage control restriction from a watermark 
embedded in the signal, wherein the watermark is 
perceptibly adapted to the signal such that the water- 
mark is hidden in human perceptible portions of the 40 
signal and the usage control restriction controls transfer 
or copying of the signal in which the watermark is 
embedded. 

2. The method of claim 1 wherein the command signal is 
represented as one or more bits in a payload of the water- 45 
mark. 

3. The method of claim 1 wherein changing operation of 
the watermark detector comprises re-programming the 
watermark detector. 

4. The method of claim 3 wherein changing operation of 50 
the watermark detector includes transferring instructions 
from a remote location to the detector to replace instructions 
stored earlier in the detector. 

5. The method of claim 1 wherein changing operation of 
the watermark detector includes changing the watermark 55 
detector in accordance with a preprogrammed rule in the 
watermark detector. 

6. The method of claim 5 wherein the preprogrammed 
rule defines a change in a watermark key. 

7. The method of claim 1 wherein changing operation of 60 
the watermark detector includes changing a watermark key 
used to decode a watermark. 

8. The method of claim 7 wherein the watermark key 
comprises pseudo-random key data. 

9. The method of claim 1 wherein changing the operation 65 
of the watermark detector includes changing how the detec- 
tor interprets one or more bits of a watermark payload. 



,285 B2 

30 

10. The method of claim 1 wherein changing the opera- 
tion of the watermark detector includes changing a response 
of a device to a watermark signal. 

11. The method of claim 1 wherein the command signal 
comprises a combination of bits of a watermark payload. 

12. The method of claim 1 including: 

changing the operation of the watermark detector with 
data provided in a watermark payload. 

13. The method of claim 12 wherein the command signal 
comprises a one or more bit trigger in the watermark 
payload. 

14. The method of claim 12 wherein the data comprises a 
watermark key in the watermark payload. 

15. The method of claim 14 wherein the watermark key 
comprises pseudo-random key data. 

16. The method of claim 12 wherein successively- 
presented watermark payloads in the signal provide the data 
used to change the operation of the watermark detector. 

17. A method for controlling operation of a device using 
a watermark detector comprising: 

receiving a media object and a command associated with 
the media object signaling that the watermark detector 
requires an upgrade to decode a watermark from the 
media object; 

in response to the command, updating the watermark 
detector to create an updated watermark detector; and 

decoding the watermark from the media object with the 
updated watermark detector including decoding a 
usage control restriction from the watermark. 

18. The method of claim 17 wherein the command is 
embedded into the media object in a watermark, wherein the 
watermark is perceptibly adapted to the signal such that the 
watermark is hidden in human perceptible portions of the 
signal. 

19. The method of claim 17 wherein the command is 
conveyed in channel different than a watermark channel in 
the media object, and the channel conveying the command 
is transmitted along with the media object. 

20. The method of claim 19 wherein the command is 
conveyed in a header of a file including the media object. 

21. A re -programmable watermark detector comprising: 
a watermark decoder for detecting a command to upgrade 

the detector; and detector instructions 
detector instructions that are replaceable in response to 
detecting the command to upgrade the detector, the 
detector instructions being operable to decode a usage 
control restriction from a watermark and control play- 
back or recording of a media object carrying the 
watermark; wherein the detector instructions control 
decoding of watermarks embedded in media signals 
and are replaceable with a different set of instructions 
for controlling decoding of watermarks in response to 
the command to upgrade the detector. 

22. The detector of claim 21 wherein new instructions are 
conveyed to the watermark detector along with a media 
object, wherein the watermark is perceptibly adapted to the 
signal such that the watermark is hidden in human percep- 
tible portions of the signal. 

23. The detector of claim 22 wherein the new instructions 
are conveyed in a watermark payload to the watermark 
decoder for decoding. 

24. The detector of claim 22 wherein the detector decodes 
watermarks embedded in media objects of a media type, and 
the new instructions are conveyed along with at least one 
media object of the media type in a channel different than a 
watermark embedded in the at least one media object. 
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25. The detector of claim 24 wherein the new instructions 
are provided in a media object file that includes a media 
object of the media type. 

26. The detector of claim 25 wherein the media object 
comprises an image signal. 

27. The detector of claim 25 wherein the media object 
comprises an audio signal. 

28. A method of encoding an upgrade trigger in a water- 
mark comprising: 

receiving a media object of a given media type; 

encoding a watermark into the media object, where the 
watermark includes a command signal used to trigger a 
change in operation of a watermark detector, the 
change being operable to change how the watermark 
detector decodes or interprets a watermark in a signal 
of the media type; and 
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encoding a usage control restriction into the watermark, 
wherein the change in the operation of the detector 
enables the watermark detector to read the usage con- 
trol restriction, wherein the watermark is perceptibly 
5 adapted to the signal such that the watermark is hidden 
in human perceptible portions of the signal. 

29. The method of claim 28 including encoding the 
command in a payload of the watermark. 

30. The method of claim 28 including encoding one or 
1° more instructions in the payload of the watermark. 

31. The method of claim 28 including: 
transmitting the encoded media object to one or more 

watermark detectors from a location remote to the one 
or more watermark detectors. 

15 
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